Supplementary material - S1: Extreme-condition tests

Introduction

This document presents the results assessing the impact of extreme condition sensitivity analysis for the Dcision Support tool for Child and Adolescence Obesity, a system dynamic model developed to explore underlying relationships that contribute to youth obesity.

Outline of extreme test sensitivity analysis

Extreme conditions test is a stress test where input parameters are varied beyond probabilistic ranges, and model-generated outcome behaviour is examined for consistency with the real or anticipated behaviour of the system (). Models that operate with logical consistency outside of “normal” conditions have greater consistency, building confidence in the model output.

Benefits of extreme conditions test:

  1. It can be used to discover flaws in the model’s conceptual logic.
  2. Aids in identifying non-linear and asymptotic behaviours that can be used to validate the model behaviour and highlight possible impactful policy levers.
  3. enhances the model’s usefulness by exploring model behaviour of systems that operate outside of contextual-historical observations, i.e. populations with differing characteristics.

Methods

A sensitivity analysis was conducted testing 649 input variables between \(\pm\) 20% of an input’s initial value. While each input value may have more extreme values, these input values are considered to be outside of probable values. For each sensitivity analysis, the model generates 78 BMI prevalence outcomes, 3 BMI categories, 13 age groups and two genders.

Elementary Effect (EE) was used to summarise the model outcome. These summaries are used to examine model behaviour. Where variables are highlighted as influential, individual input-output relationships plots disaggregate the summary results to examine behaviours further.

Outcome Measure

The plot below describes how Elementary Effect (EE) are used to summarise sensitivity analysis results.

Panel A shows an example of a model-generated outcome; over modelled time, the input assumptions (\(x\)) are varied between \(\pm\) 20% of it’s initial value. The model results at the end of the model are used to calculate the EE (panel B).

The EE is calculated for each model output result to create a sample of EE. The mean and standard deviation are used to summarise the change in outcome per percentage point shift in input assumption.

\[ EE_{i} = \frac{f(x-\Delta_{i})- f(x)}{\Delta_{i}}\]

\[ \mu_{EE}= \frac{1}{n} \sum_{i=1}^{n} EE_{i}, \hspace{1cm} \sigma_{EE}=\frac{1}{(n-1)} \sum_{i=1}^{n} (EE_{i} - \mu_{EE})^2\]

Example plot for analysis outcome

Example plot for analysis outcome

How to interpret the results

A summary plot (panel A) shows the summary relationships between variation in the input variables (overweight males body weight aged between 6 and 8) and the highlighted outcome, the prevalence of obesity.

The individual input-output relationships are plotted in the facets of panel B. Each sub-plot shows the model-generated prevalence of obesity at the end of the model for each input assumption.

As the assumed body weight for overweight males aged between 6 and 8 is varied, the plots in panel B show no impact on the prevalence of obesity for 2 and 2 to 5-year-olds, which is also reflected in the zero association in the summary plot. The prevalence of obesity for 6 to 8-year-olds and above have a negative association with 6 to 8-year-old overweight males assumed body weight. The largest associations were in the 9 to 11-year-olds, and the strength of the association tappers out for older age groups.

For every percentage point change in the assumed body weight for overweight males aged 6 to 8 years old, there is, on average, a 0.25% reduction in the prevalence of obesity in 6 to 8-year-olds. This suggests that 25% of the input variability is observed in the outcome.

Results

Overall Ranking of sensitivity

The 649 input assumptions tested can be summarised into 70 variables by taking the mean of age-gender-BMI specific inputs, resulting in 70 overarching variables. These are ranked in the following table.

Label Mean Elementary Effect
SUGAR Kj per gram 1.6110093
FATS Kj per gram 1.6028682
“NON-SUGAR CARBOHYDRATES Kj per gram” 1.4683596
INACTIVE METs 1.3562581
SLEEP METs 1.3105217
PROTEIN Kj per gram 1.0866777
SCREEN TIME METs 0.8983055
VIGOROUS PA METs 0.7847292
LIGHT PA METs 0.7593984
MODERATE PA METs 0.6088430
FATS TEF 0.3700813
CARBOHYDRATE TEF 0.3248392
PROTEIN TEF 0.3025889
Grains Reported Intake INTERCEPT 0.2382100
Dairy Reported Intake INTERCEPT 0.2266088
Discretionary foods Reported Intake INTERCEPT 0.2134536
Daily SLEEP minutes Reported INTERCEPT 0.1983803
Proportion of nutrients within food group Inputs 0.1826083
SUGAR TEF 0.1658052
Daily VIGOROUS PA minutes Reported INTERCEPT 0.1580964
Discretionary foods Reported Intake AGE SLOPE 0.1570725
Years to achieve change 0.1562435
Meat and Protein Reported Intake INTERCEPT 0.1257050
Schofield Equation Coefficient 0.1115233
Daily MODERATE PA minutes Reported INTERCEPT 0.0844130
Fruit Reported Intake INTERCEPT 0.0842263
“Body Weight (kg)” 0.0823110
Meat and Protein Reported Intake AGE SLOPE 0.0815615
Grains Reported Intake AGE SLOPE 0.0752285
Schofield Equation intercept 0.0747528
Other Reported Intake INTERCEPT 0.0576132
Vegetables Reported Intake INTERCEPT 0.0501144
Daily LIGHT PA minutes Reported INTERCEPT 0.0500310
Sugar based beverage Reported Intake AGE SLOPE 0.0497436
Other Beverages Reported Intake INTERCEPT 0.0437912
Fats and Oils Reported Intake INTERCEPT 0.0401462
Sugar based beverage Reported Intake INTERCEPT 0.0400489
Daily VIGOROUS PA minutes Reported AGE SLOPE 0.0286739
Other Beverages Reported Intake AGE SLOPE 0.0285815
Other Reported Intake AGE SLOPE 0.0251895
Vegetables Reported Intake AGE SLOPE 0.0237627
Daily MODERATE PA minutes Reported AGE SLOPE 0.0163843
Dairy Reported Intake AGE SLOPE 0.0160513
Fats and Oils Reported Intake AGE SLOPE 0.0116044
Daily LIGHT PA minutes Reported AGE SLOPE 0.0093319
Daily SLEEP minutes Reported AGE SLOPE 0.0084162
“Reference height (m)” 0.0078689
Fruit Reported Intake AGE SLOPE 0.0074201
Parents BMI 0.0048095
“Growth Function kJ/day” 0.0041664
Adult to Child Social Transmission PAL Behaviors 0.0035385
Child to Child Social Transmission of PAL Behaviors 0.0029490
Initial BMI Prevalence Inputs 0.0024548
“Percentage BF >6mths reported” 0.0023806
Initial FM % 0.0013880
Intercept 0.0012715
BMI Hazards Ratios 0.0005113
Adult to Adult Social Transmission of PAL Behaviors 0.0003405
“Percentage non-core > 0 reported” 0.0001920
Non-core >0 0.0001855
“Percentage TV >1 per day reported” 0.0001381
TV >=1 0.0001322
Breastfeeding >=6mth 0.0000777
Adult to Child Social Transmission DIET Behaviors 0.0000004
Child to Child Social Transmission of DIET Behavior 0.0000003
Adult to Adult Social Transmission of DIET Behaviors 0.0000000
Daily SCREEN TIME minutes Reported AGE SLOPE 0.0000000
Daily SCREEN TIME minutes Reported INTERCEPT 0.0000000
Water Reported Intake AGE SLOPE 0.0000000
Water Reported Intake INTERCEPT 0.0000000
Note:
Ranked variables

Body Weight

Change in Male Body Weight input assumptions

Changes in base model assumptions for body weight alter how changes in energy imbalance impact BMI category prevalence estimates, making the model insensitive to changes based on assumed energy surplus. The figure below shows the elementary effect for each local sensitivity analysis changing the assumed male body weight.

  • Changes in the body weight assumptions reduced the flow of individuals to higher BMI categories, creating higher healthy weight prevalence.
  • Younger ages impact the BMI outcomes for all older age groups.
  • Older age groups do not impact younger outcomes.
  • Changes in each BMI category mainly impact the outcome of the BMI category being changed and the neighbouring BMI category.
    • Change in the assumptions for Healthy weight impacts the underweight and healthy weight and overweight outcome.
    • Change in the assumptions for overweight mainly impacts the outcome for all BMI categories.
    • Changing the obesity assumptions change the obesity and overweight outcomes.

Change in Female Body Weight input assumptions

There are similar relationships, as noted in the male assumptions.

Additionally;

  • The results show the intergenerational effects.
    • Changes in the female assumptions slightly impact the younger age group.
  • The intergenerational effect is stronger when varying assumptions in age groups with higher fertility rates
    • For each PP increase in body weight for Underweight & Healthy weights 20 to 24 resulted in a
      • 0.57% increase in the Underweight & Healthy 20 to 24 years olds
      • 0.09% increase in Underweight & Healthy 2-year-old males (15.9% of the adults’ effect)
    • For each PP increase in body weight for Underweight & Healthy weight 40 to 44 year-olds resulted in a
      • 0.39% increase in the Underweight & Healthy 40 to 44-year-olds
      • 0.003% increase in Underweight & Healthy 2-year-old males (0.87% of the adults’ effect)

This suggests that any prevention intervention for early childhood should be targeted at age groups with higher fertility rates to have a larger impact.

Height

Similar to changes in the assumed body weight for each cohort, changes in height impact how influential energy surplus and deficits result in changes in the flows between BMI categories. However, height is the least influential in the model compared to body weight.

Change in Male height input assumptions

Change in Female height input assumptions

Growth Function kJ/day

Changes in the assumption Kj/day needed for growth resulting in relatively small changes in the BMI outcomes. The largest impact occurred in adolescent age groups where the growth assumptions where the largest. The larger observed impact in males was in 9-11 year olds and 12-15 year olds, where on average 1 Kj resulted in 0.2% change in the outcome.

Macronutrient energy density

Each food group is broken down into macronutrients; carbohydrates, protein, fats and sugars. The energy from a gram of macronutrients is an assumed model input. This assumption impacts each food group for each age-gender-BMI group.

  • The input of macronutrients becomes more sensitive for the older age group. This is due to the cumulative impact of the population’s life course.
  • Higher kJ/g leads to higher daily total dietary intake leading to a higher prevalence of overweight and obesity.
  • For every percentage point change in the assumed energy density for dietary fats, there was, on average, a 1.08% increase in the prevalence of obesity in 12 14-year-olds for females and a 1.15% increase for males.
  • This effect was 1.98% and 2.72% for 40 44 year olds females and males, respectively.

Explanation of summary results

Thermic effect of food (TEF)

The thermic effect of food (TEF) is the proportion of the energy used for digestion. A higher TEF means less dietary energy is available after digestion. TEF input assumptions impact the whole population, translating to a highly sensitivity input assumption.

  • Change in TEF, cumulative over the life course leading to a higher impact for older age groups.
  • 1 percentage point (0.0015 TEF units) increase in the TEF of carbohydrate results in a -0.06 to -0.52% change in obesity.
    • -0.44 to -3.51% change in obesity per 0.01 TEF.
  • 1 percentage point (0.0015 TEF units) increase in the TEF of Fat results in a -0.07 to -0.59% change in obesity.
    • -0.48 to -3.99% change in obesity per 0.01 TEF.
  • 1 percentage point (0.002 TEF units) increase in the TEF of Protein results in a -0.06 to -0.49% change in obesity.
    • -0.30 to -2.45% change in obesity per 0.01 TEF.
  • 1 percentage point (0.0007 TEF units) increase in the TEF of Sugar results in a -0.03 to -0.27% change in obesity.
    • -0.43 to -3.90% change in obesity per 0.01 TEF.

Explanation of summary results

Adult to Adult Social Transmission

Adult to Adult social transmission assumptions impact how changes in physical activity (PAL) and dietary behaviours are transmitted to other adults through role modelling. This assumption has little impact on the output. However, because this variable is dependent on change of behaviour, role-modelling likely interactions with intervention effects.

Adult to Child Social Transmission

Similar to “Adult to Adult” role-modelling, BMI outcome has little impact when Adult to Child role-modelling assumptions are varied.

Child to Child Social Transmission

Similar to “Adult to Adult” role-modelling, BMI outcome has little impact when Child to Child role-modelling assumptions are varied. These assumptions interact with intervention assumptions.

Infant Reported behaviours

The assumed reported infant behaviours amplify the intergenerational effects. However, these impacts are minimal.

  • The proportion of mother that breastfeed impact older age groups. This is because of the increase in energy expenditure caused by breastfeeding.
  • The proportion of infants that consume non-core foods and TV for more than 1 hr/day impacts younger age groups. Higher non-core food consumption and greater TV viewing increase the prevalence of overweight and obesity.

Assumed intergenerational relationships

Mortality ratios

Hazard ratios are applied to exogenous mortality rate so predicted population dynamics are maintained.

  • Changing these assumptions impact older age group, primarily cause by higher mortality rates in these age groups.
  • Since the primary outcome is a percentage, these ratio have little impact.

METs

Duration and intensive (METs) of physical activity are used to estimate total energy expenditure. The assumed METs for each movement category are applied to all of the population, which results in highlight sensitivity input assumptions.

  • Impact from variations in METs assumptions accumulate over the population’s life course, resulting in higher impacts for older age groups.
  • Increase in MET implies that each movement activity has higher intensity leading to a healthier population.
    • 1 percentage point increase in sleep METs results in a between -0.15% to -2.83% change in obesity.
    • 1 percentage point increase in inactive METs results in a between -0.14% to -2.81% change in obesity.
    • 1 percentage point increase in screen time METs results in a between -0.11% to -2.36% change in obesity.
    • 1 percentage point increase in light physical activity METs results in a between -0.08% to -1.63% change in obesity.
    • 1 percentage point increase in moderate physical activity METs results in a between -0.08% to -0.99% change in obesity.
    • 1 percentage point increase in vigorous physical activity METs results in a between 1.88% to 29.19% change in obesity.

Explanation of summary results

Light physical activity

Each behaviour is structured so that they are age dependent; observed surveyed behaviours were modelled using linear regression with age groups as the independent variable. This creates two variables for each behaviour; the intercept is the level of behaviour at the youngest age group (2 years old) and the change of behaviours over the life course.

  • Changes in the intercept assumptions have cumulative impacts over the life course, resulting in higher impacts in older age groups compared to younger age groups.
  • Varying age-on-age (AGE SLOPE) changes the trajectory of behaviours over the life course, making older age group BMI outcomes sensitive to changes in age slope.

Moderate physical activity

Vigorous physical activity

Screen time

Sleep

Fruit Reported Intake

Vegetables Reported Intake

Grains Reported Intake

Dairy Reported Intake

Meat and Protein Reported

Discretionary foods Reported

Fats and Oils Reported

Sugar based beverage Reported

Water Reported

Initial fat-mass (%)

Change in Male fat-mass % assumptions

Change in Female fat-mass % assumptions

Proportion of nutrients within food group Inputs

Grains

Vegetables

Fruit

Dairy

Meat and Protein

Fats

Discretionary foods

Sugar-sweetened beverages

Miscellaneous (Other)

Non-sugar-sweetened beverages

Initial BMI Prevalence Inputs

Change in Male Initial BMI Prevalence

Change in Female Initial BMI Prevalence

Plots for thesis chapter

Explaination

Example plot for analysis outcome

Example plot for analysis outcome

Males body weight 2 - 18 year olds

Female body weight including 20-24 and 40-44

TEF